Universality Laws for High-Dimensional Learning With Random Features

We prove a universality theorem for learning with random features. Our result shows that, in terms of training and generalization errors, feature model nonlinear activation function is asymptotically equivalent to surrogate linear Gaussian matching covariance matrix. This settles so-called equivalence conjecture based on which several recent papers develop their results. method proving the builds classical Lindeberg approach. Major ingredients proof include leave-one-out analysis optimization problem associated process central limit theorem, obtained via Stein’s method, weakly correlated variables.